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ABSTRACT 

A comprehensive model for planning and evaluating 
secondary vocational education programs in Georgia was developed to 
assist the state's vocational supervisors and directors in planning 
and improving vocational programs so as to make them more effective 
in enhancing students* basic and employability skills. The model is 
based on the following assumptions: educators in local schools bear 
primary responsibility for program improvement; decisions to modify 
vocational programs should be data based and related to the goals of 
the local educational agency; the Georgia Department of Education has 
a mandated responsibility for compliance reviews; annual data from 
local schools provide an opportunity for continuous monitoring of 
quality indicators; and, once available, data on programs may be used 
for multiple purposes. The four major elements of the model are needs 
assessment, context review, process components, and product outcomes. 
These were chosen because they correspond approximately to the 
variable classes in the outcome index (employment context, 
educational process, outputs, outcomes, and benefits). This guide to 
using the model begins with a discussion of the objectives of the 
comprehensive planning model, the procedures followed in developing 
the model, and compliance expectations. Program planning and 
evaluation in Illinois, North Carolina, Ohio, and Oregon are 
discussed next. The third section describes the assumptions and 
elements of the model, and the final section examines the model's 
applications to vocational education (local program questions, 
implementation of the model, and planning and evaluation issues to be 
addressed in the future), (MN) 
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FOREWORD 



The delivery of vocational education to secondary students 
may be the single most important concern confronting American 
education today. All students have a right to high-quality in- 
struction. Students in vocational courses, however, need special 
attention since the competitive strength of the nation depends on 
a skilled work force. 

The ability of the country to keep pace with international 
competitors rests with the continued output of high-quality 
products and services. Decisions concerning vocational program 
initiation, modification, and continuation affect the quality and 
level of skill proficiency. Educational agencies must be clear on 
their expectations for vocational programs, courses, and 
students . 

Statistics on enrollments, student career plans, and place- 
ments in business or industry provide relevant information for 
managing vocational education. Administrators at every education- 
al level in the local districts as well as personnel in the 
Georgia state Department of Education, need to specify program 
goals and program success indicators. The model for planning and 
evaluating vocational education programs described in this 
document provides a conceptual framework for making program deci- 
sions at any level of education. The variables may be slightly 
different, or they may receive different emphases at the state 
rather than the local level. However, the indicators of need or 
indicators of benefit should be useful for data-based analysis 
purposes at either level. Factors affecting vocational instruc- 
tion are dynamic and often subject to unanticipated change. 
Managers need a model for assessing such factors a&i student en- 
rollment trends, community influences, and student achievement. 
The conceptual model presented in this document allows readers to 
examine educational processes, take note of outcome indicators, 
and respond to anticipated needs before they become imperatives. 

Vocational supervisors and directors should find this model 
helpful in identifying courses to enhance students' employ ability 
or to teach basic skills for education after high school. The 
value of the model lies in its ability to focus attention on 
relevant areas for program improvement, such as the need to re- 
cruit students into appropriate programs — appropriate not only for 
the student, but for the job market as well. 



iv 



I 



Appreciation is extended to persons who have reviewed this 
docuinent--to Dr* Morgan Lewis, Dr* McCaslin, Dr, Floyd 

McKinney, and tv> Dr* Harold Starr, all of the National Center, 
pr* Starr deserves a special note of appreciation because his 
ideas and research studies formed the basis of the conceptual 
framework* in addition, we gratefully acknowledge the 
contributions of those who provided information from the states* 
They were Dr. Donald R, Brannon, North Carolina Department of 
Public Instruction; Mr, John Klit, Illinois State Board of 
Education; Mr, George Sterling, Ohio Department of Education; and 
Mr* Tom Williams, Oregon Department of Education* 



Ray D, Ryan 

Executive Director 

The National Center for 

Research in Vocational Education 



ERIC 



^ 6 



!• INTRODUCTION 



This model began with a request from the Georgia Department 
of Education to develop a process model for planning and evaluat- 
ing vocational education programs. The request grew out of an 
educational reform movement in Georgia that produced the Quality 
Based Education (QBE) Act of 1984 and a spirit of inquiry into new 
ways of delivering vocational education • Vocational education is 
coxnmonly defined as organized programs, services, and activities 
which are directly related to the preparation of individuals for 
paid or unpaid employment, or for additional preparation for a 
career requiring other than a baccalaureate or advanced degree 
(Center for Education Statistics, 3, n^dO- This model is a 
response to the desire for high-quality vocational education. The 
model is predicated on a belief in the value of data-based manage- 
ment decisions. 

The National Center drew on a long history of involvement in 
vocational education programming to develop this model. Chief 
among these activities was a data-based management approach cre- 
ated by Harold Starr (1986). This approach describes a method of 
using empirical data to i-nprove planning and evaluation decisions. 
In particular, the outcomes index inherent in these methods allows 
comparisons among vocational programs and occupations. Anteced- 
ents of the outcomes index can be found in early reports from the 
National Center (Starr, Merz, and Zahniser 1982). Context, out- 
comes, and benefits elements can be found in these reports. Other 
National Center documents influencing the development of this 
model are the following: Brannon (1985); Campbell and Panzano 
(1985); Darcy (1979); Farley et al. (1985); Franchak (1983); 
McKinney (1985); and Stevenson (1979). Also, information was 
obtained from several states in an effort to assess the state-of- 
the-art practice in collecting planning and evaluation data. 

Programs from four states — Illinois, North Carolina, Ohio, 
and Oregon — are summarized in chapter 2 of this document. The 
state programs were examined to gain insights into the best 
methods of program planning and development. The conceptual model 
presented on tho following pages contains fundamental elements of 
planning and evaluation embedded in a flow chart of relationships 
and processes. These concepts are described in chapter 3. 
Chapter 4 contains applications of the model to vocational 
processes. Some of these applications are unique to Georgia. 
Finally, issues and unresolved problems associated with the use of 
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the model are summarized under the heading "Next Steps" in 
chapter 4. 



Objective^ 

The objectives for this deliverable are as follows: 

o To develop a comprehensive vocational education program 
planning and evaluation conceptual model 

o To describe its application to vocational education in 
Georgia 

The model described in this document is applicable to local, 
regional, and state levels of secondary vocational education. 
The focus of this report, however, is at the state level. The 
assumptions and values implicit in the model must be adopted 
statewide if the information gathered from local schools is to 
have meaning as aggregated data. Implicit in this model are two 
key elements? the outcomes index and on-site reviews. The index 
serves as a triggering device for on-s^ite reviews, in that sense, 
the outcomes index monitors the quality of vocational programs 
annually in an accountability mode. The on-site review, on the 
other hand, examines processes associated with the management of 
secondary vocational education instructional programs. The on- 
site review is for program improvement of in-place programs or for 
review of new programs. 



Procedures 

The comprehensive process model was developed following a 
series of interviews with state department staff in Georgia, 
observation of a Dekalb County vocational programs evaluation, 
conversations with persons in other state departments of educa- 
tion, and a review of documents in the Educational Resources 
Information Center (ERIC). All of these activities contributed to 
the development of the model; however, the discussions with 
the Georgia staff were most helpful. 

This comprehensive program planning and evaluation model is 
predicated on the following principles: 

o Excellence is achieved not by meeting minimum standards 
for compliance reviews, but by obtaining quality informa- 
tion for use by dedicated personnel to meet identified 
student and community needs. 



ERIC 



2 

8 



o The Georgia Department of Education plays a leadership 
role in vocational program improvement. 

o Program improvement site visits to local schools should be 
driven by program need, not by chronological time. 

The following guidelines were used in the development of the 
model : 

o To the degree possible, data available from the planning 
and evaluation model should be compatible with the depart- 
mentwide GENESIS project. 

o Program improvement should be dynamic and driven by local 
strategic planning. 

o Recorded vocational program data, transmitted within the 
state, should be accurate, reliable, and easy to under- 
stand . 

o The time lag between the collection and report of data 
should be minimized. 

o Program performance indicators must be relevant to plan- 
ning and evaluation indicators. 

o Data produced by the implemented model with its outcomes 
index and program improvement reviews must be accessible 
to those who need it. 

o Data on students, equipment, staff, and other variables 
must be in a form to be aggregated across classes to yield 
measures for programs. 

These guidelines are based on characteristics of data desired by 
the Center for Education Statistics (1986) . The characteristics 
of these data follow: 

o Accurate. Given access to the same information in the 
school district, different individuals should be able to 
provide the same data on the data request sheet. 

o Comparable. Concepts such as "program completers" should 
be comparable within schools and across school districts. 

o Timely. Data should be available when it is needed and in 
a form that facilitates its use. 

o Easily Obtained. Minimal respondent burden should be 
necessary to provide the information requested. 
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Two cautions are noted when using the model — 

o A time lag will occur between the time when suggestions 
for program impr-»vements are made during the on--site 
review and the time-related changes show up in the out- 
comes index. The time lag will occur because process 
factors often are only indirectly related to program 
outcomes. For example, time is required to influence 
placement rates or performance skills. 

o The coding of the variables for the index may not always 
sxibscribe to orthodox measurement assumptions. For exam- 
ple, it may be necessary to code a basically dichotomous 
variable 1, 2, 3 in order to reflect a site history of 
compliance for the index formula or some other unique site 
situation. 



Compliance Expectation s 

The advent of the Quality Based Education (QBE) Act raised 
expectations for high-quality vocational education in Georgia. 
Performance coupled with accountability seems to be the current 
emphasis in education. New performance standards are being writ- 
ten for all of education, and student mastery of competencies in 
core courses as a graduation requirement is desired. 

Vocational education courses at the secondary level are 
receiving much the same treatment as other subjects at secondary 
and elementary levels. Standards are being written and funding is 
being placed on a schoolwide full-time equivalent (FTE) basis. 
Tjie burden for planning specific vocational programs and courses 
will fall on local supervisors and teachers. Decisions to approve 
or modify these programs and courses will continue to be made by 
the state staff according to overall priorities in the state. 
This combination of local interests and state perspective must be 
nurtured to reflect equitable decision-making responsibilities. 
In this environment, a premium is placed on relevant, high quality 
data for planning and evaluating programs. State data managers 
have an excellent opportunity — and responsibility — to influence 
decisions with reality-based data. Of course, the data must be 
timely and in a form that can be used by appropriate persons. The 
collection, an alysis, and transmission of evaluative data for 
managing vocatio nal programs is one of the greatest needs of 
education in Geo rgia at the present time . Most of the responsi- 
bility for enforcing the requirements of QBE will fall on the 
Standards and Evaluation Division of the Georgia Department of 
Education. 

At the federal level, the data requirements for compliance of 
vocational programs are still evolving following the demise of the 
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original Vocational Education Data System (VEDS) . However, it 
appe.:4rs likely the states will be asked to provide very little 
data. Public Law 98-524 no longer requires specific enrollment- 
based data, except for section 423 which is concerned with the 
access to vocational-technical education programs by handicapped 
secondary school students. Section 423 does require information 
on total handicapped enrollment by program at the 4-digit classi- 
fication of Instructional Programs level, by type of instructional 
setting, and by type of handicapping condition. (A prudent state 
administrator would collect the same type of information for 
disadvantaged students as backup for surveys done by the National 
Assessment of Educational Progress.) 

A pressing need at this time is for data definitions. No 
agreed-upon definitions exist. Without definitions, no meaningful 
data can be aggregated across state lines. An important clue to 
federal data needs may be found in the type of information being 
collected by the National Assessment of Vocational Education 
(NAVE), 1986. The NAVE study plan indicates an interest in a 
state's capacity to address the priorities identified in the 
Perkins Act. NAVE's studies will focus on special populations and 
on conditions affecting secondary and postsecondary vocational 
education. The nation's changing economic skill requirements 
remain a priority for all education and training systems. The 
mandate for the National Assessment calls for descriptions and 
evaluations of the following: 

o The vocational education services being delivered to 
special populations 

o The effects of the act on modernizing the nation's voca- 
tional education system and meeting the changing needs of 
the workplace 

o The resources required to meet adequately the nation's job 
training needs 

o The impact of vocational programs on the academic skills 
and employment opportunities of students 

o The coordination of vocational education programs with 
employment training and economic development opportunities 
in the states 

o The coordination of vocational education programs and 
services available for students who are handicapped or 
disadvantaged 

o The skill levels and occupational competencies developed 
by the states to assess their vocational education pro- 
grams 
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o The effectiveness of vocational education programs for 
individuals with limited English proficiency 

o The effectiveness of bilingual vocational and instructor 
training in meeting the needs of adults with limited 
English proficiency (see attachment A) 

It appears that most of the national data needs will be met 
through on-going data collection activities of the Center for 
Educational Statistics such as the National Assessment of Educa- 
tional Progress and the longitudinal studies. States have few 
obligations for providing enrollment data to Washington on a 
routine basis. In fact, compliance guidelines for the Carl D. 
Perkins Vocational Education Act have not been issued to date. 

The proposed plan for Data on Vocational Education (DOVE) , 
authorized by the Perkins Act, differs from VEDS in several ways. 
First, the system relies on sampling rather than surveying the 
providers of vocational education programs. Second, the national 
collection is no longer confined to using data collected by the 
states. Third, the new legislation gives wide discretion to the 
secretary of education in determining the scope, structure, and 
frequency of data collections. 

The data needs of vocational education at the national level 
appear to be broadly defined in relationship to general education. 
Enhanced elementary and secondary common core data elements are 
being field tested in 10 states at the present time. These teach- 
er and school suxrveys contain questions about vocational educa- 
tion. Following the administration of these questionnaires in 
1988, student transcript data will be collected from the same 
schools in 1989. This nationally representative sample of teach- 
ers and schools will be considered the primary source of data on 
what students know and can do. other sources of data nationally 
are surveys by the National Assessment of Educational Progress 
(NAEP) and the national longitudinal surveys. The NAEP surveys, 
in some cases, will allow data for states to be disaggregated and 
compared; h«pwever, this data is not likely to be present in suffi- 
cient quantity to meet state planning needs. 

Data elements for determining baseline measures and the speed 
of progress toward educational goals are dependent upon the under- 
lying values and vision for the populace of a state, with this in 
mind, the comprehensive planning and evaluation model put forth in 
this document has been constructed using the values and assump- 
tions inherent in the Quality Based Education Act of 1984, the 
enabling legislation for public education in Georgia. 
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2. PROGRAM PLANNING AND EVALUATION 
ACTIVITIES IN OTHER STATES 



State responses to the changing federal compliance requests 
have been varied. Some have placed more responsibility for plan- 
ning and evaluation decisions at the local and district levels. 
Others have maintained strong centralized data collection and 
processing capabilities. Most have kept the Vocational Education 
Pata System (VEDS) questions in one form or another. Responses to 
these questions have filled an information need in the states for 
accountability data. Four state planning and evaluation systems 
are described on the following pages. They are summarized in a 
table at the end of this chapter. 



Illinois 

The state board for vocational education in Illinois is 
midway in the process of implementing a new plan entitled Educa- 
tion for Employment. It will not be fully implemented until 1988, 
and it represents only the initial stage of a comprehensive plan- 
ning and evaluation system. The goals of quality, efficiency, and 
expanded opportunity for students to enroll in vocational educa- 
tion drive this new plan. The plan features the identification of 
61 regions in the s*^'^te to serve as a support system to the 750 
local high schools in Illinois. The existing nine vocational 
administrative districts are superimposed over these regions. 
Each district contains a state staff member who is responsible for 
vocational education. The staff member works with the regions 
within his or her district. These regions cut across school 
district lines and are the vehicles for planning and evaluating 
vocational education programs. 

An advisory committee is appointed for each program in each 
region; with the advice of these committees, annual applications 
for new and existing programs are tentatively approved at the 
regional level. Over 100 programs are possible for approval, but 
each program can be classified into 1 of the 5 traditional ser\'ice 
areas of agriculture, business, industrial, health, and home 
economics . 

The use of site visits to review programs is in the pilot 
stage. Currently, a committee comprised of stat^ staff, peer 
review teachers from neighboring regions and cersons from the 
community spend time on site reviewing programs in each of the 61 
regions. The regions are likely to experience these reviews on a 
4 -year cycle. Data on enrollment trends, costs, student satisfac- 
tion, employer satisfaction, and placement will be made available 
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to the site team prior to the visit. Assessment of student compe- 
tencies is planned for employability skills and technical skills, 
but this assessment is not intended to be uniform for the entire 
state. This last point is consistent with the principle of local 
autonomy. There is no state curriculum in Illinois. However, the 
State Board of Education has supplied each of the regions with 
lists of tasks related to generalizable employability skills. A 
great deal of labor market information is made available to the 61 
regions. Enrollment is interpreted by committees in the dis- 
tricts. Task lists for specific occupations are verified in each 
of the 61 regions by incumbent workers. Currently, these occupa- 
tional task lists are then transformed into courses and vocational 
programs. New courses are determined by the regions based on 
labor market information, local verification of task lists, and 
student interest in the courses. There are no maximum or minimum 
enrollments for each class. This is determined by the local 
school district. 

Records on student progress will be maintained at the local 
level and used by the region to plan courses. For example, each 
district defines the term program c ompleters as students who have 
completed a sequence of courses in one or more vocational pro- 
grams. Data on program completers are collected by the state 
office. The questionnaire is sent to the local region for a 6- 
month follow-up of local program completers. Disadvantaged, 
limited English proficiency, anc! handicapped students also are 
followed-up in the same manner. In the future, computers in the 
region will be used to transmit local data to a state-managed 
database. Also, a 3- to 5-year follow-up of program completers is 
in the planning stage. It is important for a state to have a 
comprehensive planning and evaluation system. Fragmented data can 
sometimes give misleading information. One of the primary uses of 
evaluative data on vocational programs is accountability to tax- 
payers as well as program planning. 



North Carolina 

The two increasingly influential factors in determining 
occupationally orientated vocational education programs in local 
schools in North Carolina are student demand and completer employ- 
ment rates, as specified by the North Carolina General Assembly. 
Job skills courses now must be justified annually based on unem- 
ployment rates for the county for the latest 2 available years. 
These rates for 16- to 19-year-olds are determined by the Employ- 
ment Security Commission from a formula supplied by their office 
in Atlanta, Georgia. The unemployment rates for the vocational 
job skills course completers for the last 2 available years must 
be lower than the youth unemployment rate for the county. Pro- 
grams are allowed 2 years to phase out when no job need exists; 
cooperative programs may be maintained up to 3 years to increase 
their enrollment or redirect efforts due to duplication. The 
vocational program completer unemployment rate for the state is 
about 6 percent (higher for completers of special needs programs) , 
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but the unemployment rate for youth statewide is more than double 
this rate. 

Following a local control philosophy, local plans for voca- 
tional education programs are submitted to regional program coor- 
dinators for initial approval prior to formal approval by the 
state board. Applications for programs must meet the standards 
required by the 1985 General Assembly. These standards include a 
minimum of 3 programs per school and at least 12-18 students per 
class as interpreted by the Revised Vocational Education Program 
of Studies. 

Vocational education in North Carolina is to be available to 
all students who desire it in the public schools as an integral 
part of the educational process. As in many other states, voca- 
tional education is defined as prevocational, introductory 
courses, consumer and homemaking, industrial arts, and preparation 
for advanced education, as well as skill development. Only the 
occupationally oriented courses are required to meet the youth 
unemployment rates of the county. However, cooperative vocational 
classes must meet special criteria. Eighty percent of the cooper- 
ative program completers^ must be placed in occupations related 
to the content taught in the classroom. Twenty percent of the 
local educational agencies (LEAs) in each of the eight regions are 
randomly selected yearly to submit forms on the cooperative pro- 
gram to regional vocational coordinators. Follow-up of vocational 
job skills course completers is conducted annually by the state 
department of education in the spring. Local education agencies 
complete a paper form that is mailed to the state office. The 
state is moving toward an electronic network that allows data to 
be entered at the local education agency. Editing of programs 
should occur at the local level where students and classes can be 
f ollowed-up . 

Each vocational program is required to have an advisory 
committee comprised of employers and others who represent the 
occupational clusters and non job content within a program area. 
The number of members is determined by the size of the program 
within the local educational agency. A separate committee, a 
Vocational Education Improvement Council (VEIC) (Brannon 1985) , 
functions within each LEA undergoing the federally mandated pro- 
gram review in the manner of a quality circle to provide input 
into the improvement and expansion of the 2-year local plan. The 
VEIC serves the complete local school administrative unit (LSAU) . 
In addition, program area improvement task forces exist for each 
program area in the LSAU. The chairperson of each program area 
task force serves on the VEIC. Serving on this council are teach- 
ers, business representatives, and administrators. 



^Program completers are those students who have satisfactorily 
concluded at least a second course in a sequence of courses for a 
secondary vocational program. 
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The work of these local committees provides the primary basis 
for on-site reviews by state department staff and others. Most 
orten these visits are invited by the locals. The visits usually 
are associated with the development of a new program or the review 
of a new teacher. 

Vocational program reviews are scheduled once every 5 years, 
bttt these reviews focus on documentation of program improvement 
activities. The reviews are conducted by state staff by the mail 
and telephone. Generally, site visits for individual programs or 
teachers are not used for these reviews. Reviews usually consist 
of determining the congruence between data and the local improve- 
ment planning generated by the program area improvement task 
forces (by state staff program area specialists) and the local 
planning generated by the VEIC (by the regional coordinators) . 



Ohio 

,rrr,^rl^^ 2^^^ Pr«pgram for Improvement, Development, and Expansion 
(PRIDE) of vocational education features a centralized information 
system with interconnected linkages to other state agencies the 
Ohio Bureau of Employment services, for example, orientation 
meetings for school district administrators in vocational planning 
districts are held in December as part of the Local Education 
Agency Plan (LEAP) process to plan for the next school year. The 
vocational planning districts are coterminous with the Department 
of Labor's Service Delivery Areas to facilitate use of labor 
market information. In January of each year, local district 
administrators submit a plan for vocational programs to be offered 
in the fall of the next school year. This plan and the subsequent 
reimbursement of the programs are adjusted to actual class 
enrollments. The PRIDE and the LEAP processes complement one 
another for program improvement purposes. 

The PRIDE process is conducted in each school district every 
5 years. The review schedule is known many years in advance. The 
district appoints a self-review committee community comprised of 
educators, community members, and others to examine a vocational 
program's curriculum, instructional process, facilities, equip- 
ment, staff, and assessment of students. Former students and 
employers are included among this local self-review committee. 
This committee's recommendations are read and commented on by a 
state supervisor. The supervisor makes a visit to the school, 
advises the teacher and district administrator of the changes 
needed, and files a report with the state board and the school 
district. A district has 5 years to make up deficiencies in 
programs without loss of reimbursement. Currently, Ohio reim- 
burses for 12 vocational program fields, including adult vocation- 
al education, at the secondary level. All of the program fields 
in one district receive visits from state staff during the one 
week of review. This contains any disruption to one week during a 
5 year period. ^ 
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Programs must meet strict guidelines to qualify for cost 
reimbursement. These can be no more than 25 but at least 15 
students in each junior class. Enrollments are sometimes relaxed 
for seniors. Programs must have at least 60 percent placement of 
program completers2 in related occupations, and an unemployment 
rate of program completers that is at least 10 percent better than 
the prevailing employment rate for youth aged 16-21. For example, 
if the unemployment rate is 19 percent, then the allowable number 
of unemployed vocational program completers would be no more than 
10.9 percent. Graduates who are going on to postsecondary educa- 
tion or military service, or those who are looking for work are 
excluded from the computation of ♦•hese statistics. The Ohio 
standards also tie the availability of vocational programs to the 
percentage of graduates entering college or degree-granting high 
education programs. As the percentage of college enrollments 
increases, the percentage of students enrolled in vocational 
education is allowed to decrease. Local school districts that 
fail to meet the program completers placement criteria are allowed 
3 yeara to improve their performance. 

Programs are initiated on anticipated demand in an area. 
There must be a need for 25 workers in an occupation per year in 
order to offer a program. Districts may either conduct their own 
surveys to establish this number or accept the Department of 
Labor's statistics. Another approach used is an agreement between 
two contiguous districts to combine their employment needs, allow- 
ing one district to offer the program. 

Questionnaires are mailed by the state department of educa- 
tion to collect 1-year and 5-year follow-up information. Machine 
readable forms were considered and rejected because of problems 
with accuracy. Respondents had difficulty completing the forms. 
Currently, school-based data are filled in on program completers 
by district administrators, one form per class. The form is 
transmitted to the state department where it is keypunched into 
magnetic tape and prepared for analysis. 

The strong centralized accounting procedures allow easy 
access to records for state accountability and planning purposes. 
The relatively strict guidelines for reimbursement may tend to 
limit local offerings of creative programs except for experimental 
pilot activities. ^ 



Oregon 

The Planning for Progress vocational education planning and 
evaluating documents developed by Oregon state University in 1981 
have been used with success by local school districts; however, 
budget constraints at th e state level have forced changes. This 

2a program completer is a student who has successfully completed 
a series of classes in at least one program area. 
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process took 6 months of lead time at a cost of approximately 
$3,000 per medium-sized district. Currently, some local districts 
are continuing this process as members of the Alliance for Program 
Improvement at Oregon state University. New procedures for pro- 
gram reviews have been scaled down to fit available resources. 
The Planning for Progress procedures are not required by the 
Oregon Department of Education. 

Current procedures call for periodic reviews of program 
goals, processes, and outputs in one of 16 regions. The Office of 
Civil Rights (OCR) reviews and the program planning reviews occur 
at the same time. Three sequential meetings are held. The first 
is a meeting between the state department representative and the 
regional coordinator. The second involves a meeting with the 
state department representative and a team of representatives from 
the respective secondary school district. Usually, this team is 
comprised of an administrator, an instructor, and an advisory 
council member. Records from the school are reviewed, and deci- 
sions are made on program options; final approval of programs is 
reserved for the state staff. It usually takes 6-8 weeks between 
the two meetings. The reviews occur once every 5 years for each 
school. The third meeting is held upon completion of the evalua- 
tion to review findings and discuss planned activities to 
strengthen program operations. The same district teams are in- 
volved in the last two meetings. 

Programs are approved based on the match with labcr market 
data. These data most often come from the state for the standard 
18 cluster areas or from local surveys for experimental programs. 
One occupational database in the Employment Security Office serves 
needs in vocational education. Job Training Partnership Act (JTPA) 
programs, the Department of Economic Development, vocational 
rehabilitation, and other areas. 

Reimbursement of program costs are based on a weighted stu- 
dent credit hour. That is, a weighted student equals one student 
for 1 year for 2 hours a day. Most of the vocational classes are 
offered for 2 hours a day, 5 days a week, exceptions, such as a 
student who takes only one semester of the class would receive a 
0.5 weighted credit. Cooperative program students receive an 
additional 0.5 credit per year; 0.5 credit also is given for 
participation in youth organirjation activities. Therefore, it 
would be possible for a vocational student to earn two credits per 
year if he or she were in a co-op program and active in a student 
organization. Student records are completed on forms at the local 
school and mailed to the state office to be processed. Larger 
districts transmit student records on data tapes. This accounts 
for approximately one-third of the records at this time. 

Advisory councils are required for each program in the school 
district. They play an active role in the design and implementa- 
tion of the skill development programs. Coordination with 
postsecondary institutional programs are encouraged; this articu- 
lation is indicated in the local plans. Program variations are 
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approved on an as needed basis. Variations seem to be tending 
toward shorter-term programs due to problems in scheduling two- 
period classes. 

. ^J^^ addition to the program planning reviews, a team of state 
staff reviews assurances and reports, and if problems are de- 
tected, visits the school district to determine if the school is 
in compliance with state standards. Vocational education state 
staff serve with others in this compliance audit. 

Formerly, students were followed up during the first 6 months 
following graduation, 3 years after graduation, and in a longitu- 
dinal data base. The 3-year study and the longitudinal surveys 
used samples of students. The samples were usually too small to 
be used in a reliable manner. The limited budget, limited time, 
and the diversity of the districts— geographic location and size- 
tended to work against the collection of reliable data. The 
follow-up surveys other than the 6-month study were discontinued 
due to lack of funds. The follow-up surveys were conducted by the 
local school district using state forms. A nominal grant to the 
local district allowed staff to invest time in finding out what 
happened to students. This normally resulted in a higher percent- 
age of returns than forms mailed from the state office. Although 
the amount of data being collected for program planning and evalu- 
ation purposes in Oregon at the present time is less than a few 
years ago, the programs are better coordinated than they have been 
due to the activity at the district level. More versatility is 
also provided in the data as machine-scoring increases. 



Summary 

A cursory review of data collection forms from these states 
and others shows a number of 1986 and 1987 publication dates. 
Many states are changing their data collection procedures to take 
advantage of new information processing technology and the flexi- 
bility allowed under the Perkins legislation for meeting data 
needs. Florida, for example, has developed a computer-based 
student follow-up network for transmitting information from local 

^° ^^^^^ department of education. As much as one- 
third of the data is transmitted electronically. States are 
moving toward the student full-time equivalent (FTE) indicator as 
a measure for reimbursing the cost of a vocational program. 
Others, such as Arkansas, are going to a student credit hour 
indicator for reimbursement. This freedom to plan is placing more 
and more responsibility at the local level. As these measurement 
systems develop, it is becoming increasingly clear that states are 
using cost containment as one of several criteria for evaluating 
vocational programs. 

These four states represent different approaches to planning 
and evaluation decisions in vocational education. They are geo- 
graphically distributed throughout the country. The following 
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table lists selected features of these planning and evaluation 
systems . 



Despite a common definition for a vocational education pro- 
gram, the states have elected to apply this definition differ- 
ently. Oregon and Illinois, for example, are building programs 
around families of occupations. Ohio and North Carolina, on the 
other hand, have tended to expend the number of program offe*.- 
mgs. 

All states require an advisory council for local programs. 
In most cases this is one council for each vocational program. 
The councils are viewed as a primary means of ensuring that the 
skill-training courses are relevant to the employment needs of the 
communities, in a few cases, the councils operate at a regional 
level and offer advice about the needs of programs across school 
districts within their region. 

Requirements for minimum and maximum student enrollments in 
courses are left up to the local districts in Illinois. Ohio and 
North Carolina require minimums in vocational classes; however, 
these may be relaxed somewhat for senior classes. 

Consistently, states are requiring labor market data to 
3ustify job skill programs. Sometimes these data may be collected 
locally among employers to indicate the need for a new program, 
but more often they come from employment security offices at the 
district level. Of course, the number of program graduates placed 
in related occupations is the ultimate indicator for job skill 
course continuation, other placements, such as military service 
and advanced schooling, are counted as successful placements for a 
course. Typically, school districts have up to three years to 
improve their placement rates. 

Site visits by outside review teams continue to occur. The 
frequency of team visits varies somewhat, but consistent with the 
Perkins Act, most school districts receive a visit at least once 
every five years. The direction of change, however, is toward 
fewer site visits, in Illinois, the site visit occurs at the 
district level when advisory committee are formed for rural, less 
populated sections of the state. There must be a need for a visit 

■ ^® ^^^^ North Carolina or Oregon. In Ohio, the 

visit is done by one supervisor for each of the vocational pro- 
grams. As information on local school programs becomes more 
available, routine site visits will be fewer. 

The states contacted were all following up their program 
completers within the year following graduation. Sometimes this 
occurred in the fall; most often it was late winter before the 
forms were returned and summarized. Ohio sends this survey by 
mail from the state office directly to the program completers, 
other states, such as Illinois, rely on local school officials to 
maintain contact with students after they graduate from high 
school. The reasoning seems to be that school officials closer to 
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TABLE 1 



SELECTED FEATURES OF VOCATIONAL "PROGRAM PLANNING AND EVALUATION SYSTEMS IN FOUR STATES 



System Characteristics 


Illinois 


North Carolina 


Ohio 




Number of vocational 
programs 


100 plus programs in 
five areas based on 
occupaticMis 


8 


12 


18 clusters 


Use of advisory 
conmittees 


Yes 


Yes 


Yes 


Yes 


Min-Max student 
enrollment for 
existing programs 


Hone 


12-18 


15-25 


Notc 


Criteria for starting 
new programs 


Labor maricet demand, 
local verification 
of need, student 
interest 


Student demand and/ 
or labor market 
demand (there must 
be three programs 
per school) 


Annual demand for 25 
persOTS in the 
occupation 


Labor maricet and student 
interest 


Time allowed for 
redirection (in years) 


Determined by local 
planning coraoiittee 


2-3 


3-5 


60 day follow throu^ 
plus up-to-1 year for 
correct icxi 


Use of site visits 


Yes 


Yes, when need is 
demonstrated 


Yes 


NcMie, other than the 
standards visit 




ouaue Starr and 
peers as needed 


Regional coordinatcr 
and others as 
necessary 


State supervisor 


State supervisor, local 
administrator, teacher, 
and advisory council 
member at the district 
meeting 


Vocational student 
follow-up 


1 year 


1 year 


1 year and 5 years 


1 year 
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the student are in the best position to find out what he or she is 
doing after graduation. 
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3. DESCRIPTION OF THE CONCEPTUAL MODEL 



This chapter contains the assumptions of the model, and a brief 
review of the main headings in the model. 

Assumptions of the Model 

This model contains categories of variables intended to 
represent major elements of a program planning and evaluation 
process. It is conceptual in nature; the items within the cate- 
gories are exemplars of the major headings. The concepts at all 
levels of specificity represent variables to be measured or ad- 
dressed at some point in the implementation of the model. The 
following assumptions provide a frame of reference for the model: 

o Program improvement is a responsibility of vocational 
educators at all levels, but educators in local schools 
bear primary responsibility because education is fundamen- 
tally a local enterprise . 

o Decisions to start, stop, or modify vocational programs 
should be data-based and related to the goals of the 
educational agency. 

o The Georgia Department of Education has a mandated respon- 
sibility for compliance reviews. 

o Annual data from local schools provide an opportunity for 
continuous monitoring of quality indicators. 

o once available, data on programs may be used for multiple 
purposes . 

Program evaluation occupies center stage in this document 
because of the QBE emphasis on quality and accountability in 
education. Standards for vocational education are being revised 
and developed at the present time in Georgia, state staff need to 
identify variables useful for evaluating vocational programs. 
Performance indicators for students and programs must be quantifi- 
able and verifiable for use in the outcomes index. Current expec- 
tations call for this information to be made available to program 
reviewers prior to program improvement site visits. All of the 
programs would be reviewed approximately once every five years. 
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The planning and evaluation model described in this report identi- 
fies some of the variables to be examined by the site team during 
their visit. The variables in the model also represent candidates 
zor the outcomes index. 

The site team visits are expected to dwell on program im- 
provement activities. Most likely these visits will focus on 
educational processes necessary to the development and operation 
of a high quality vocational program. Some of the processes are 
student counseling associated with enrollment in vocational 
courses, curriculum development and improvement, and follow-up of 
vocational program completers. 

A self study report would be developed prior to the site team 
visit. This report would document, among other factors, the needs 
in the community for vocational program completers. Activities 
leading to the planning and development of local vocational 
programs should have preceded the self study. The active involve- 
ment of advisory councils, for example, can be extremely useful in 
specifying the needs in a community. 

Some of the same variables in the model can be used for 
planning and/or evaluation activities. The participation of 
program completers in the labor force, for example, provides 
valuable feedback on the use of occupational skills acquired in 
the courses, and it reflects the availability of occupational 
openings in the community— an important planning variable. Local 
planning committees, such as the ones currently operating in North 
Carolina, should be used to anticipate needs of students and 
employers . 

The vocational education program planning and evaluation 
model illustrated in figure 1 contains process and product vari- 
ables. It includes context and needs factors. These broadly 
based concepts can be applied at the state, regional, or local 
education levels. Factors such as the diversity of local educa- 
tion agencies make application of the model more difficult at the 
state level than at the local level. The question of model 
application will be addressed in the next chapter of this report. 
Although this pro:,ect concerns itself primarily with state-level 
planning and evaluation, the model has applications at the local 
level . 

The four major elements of the model— needs assessment, 
context review, process components, and product outcomes— corre- 
spond approximately to the classes of variables in the outcomes 
index— employment context, educational process, outputs, outcomes, 
and benefits. Elements in the model form categories for variables 
ii ^® selected as indicators in the index. Quality Based 

Education (QBE) standards and compliance expectations of the 
Perkins legislation represent influences acting upon educational 
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Vocational Education 
Needs Assessment 
(Values Driven) : 



Individual 
•Sex 



•Race 
•SES 

•Handicapping 
condition 



•Disadvantaged 

•Marital status 

•Interest 

•Knowledge 

•Attitudes 

•Skills 



Employers 

•Size of employer 
•Typs of employer 
•Level in the company 
•Knowledge^ skills, and 

attitudes required 
•Occupational mix 
•Present and future 

employment requirements 

Society 

•Educational 

•Economic 

•Social 



Educational Environment Context Review ; 

•Community expectations for skill development 
•Availability of jobs 
•Populations trends and issues 
•Availability of education and training beyond 
high school 



Educational Process Components ; 





Program Initiation 
Program Operation 
•Goals 
•Schedules 
Program Termination 





QBE 
Standards 



A 



Perkins 
Legislation 



Educational Product Outcomes 
(Benefits) : 

Economic Indicators 

•Labor force participation 
•Employment and unemployment 
•Training-related placement 
•Type of employment 
•Nontraditional employment 
•Earnings 

•Employee satisfaction with work 
•Employer satisfaction with employee 
•Cost-effectiveness 

Knowledge and Skills 

•Basic educational skills 
•Information-seeking skills 
•Interpersonal skills 
•Problem-solving skills 
•Employability skills 
•Knowledge of the world of work 
•Occupational skills 

Personal Characterfstlcs 

•Aspirations 
•Attitudes and values 
•Self-esteem 
•Citizenship 
•Sense of efficacy 
•Satisfaction with education 
•Sense of independence 
•School attendance and dropout 
status 

•Continued education status 



Rgure 1. Program planning and evaluation conceptual model 
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processes at all levels. The major elements of the model are 
intended to portray a self -renewing system with processes designed 
to support the flow of information to decision makers. 

Process data, such as information about employers, can be 
used to assess needs in the local community. Not only can future 
employment requirements be estimated, but the size and type of 
business can be noted as information for curriculum changes in the 
classroom, other process data bearing on the issue of program 
development are population trends in the community; the avail- 
ability of education and training beyond high school; and the 
availability of facilities, equipment, and instructional staff. 
Product data most often relate to students. Data on students' 
Icnowledge, interests, and attitudes are helpful when counseling 
them to enroll in specific vocational courses. Data on students' 
knowledge, attitudes, and skills acquired in the vocational 
classes are indispensable to program evaluation. The educational 
product outcomes category in the model also contains personal 
characteristics of the student; these variables greatly influence 
the acquisition of skills for the workplace. The economic indica- 
tors in the outcomes category are product-related measures of 
vocational education, but they are one step removed from direct 
measurement of student skills. The economic indicators are influ- 
enced by such factors as the opportunity for employment in the 
area. 



Elements of the Model 

Needs assessment is espential to sound program planning and 
evaluation. It affects the decision to offer new classes and how 
these classes relate to employment opportunities. A valid assess- 
ment can identify youth and adults in need of vocational educa- 
tion. It can pinpoint knowledge and skills to be taught and 
suggest ways of teaching these students. 

A context review can help identify community expectations for 
skill development. Some communities, for example, send a higher 
percentage of their graduates to postsecondary education than 
others. Out-migration of population or a sudden influx of new 
businesses can change the need for specific skill instruction 
dramatically . 

Process components of the educational system can make or 
break vocational programs. Administrative support for state-of- 
the-art equipment can result in graduates with competitive skills 
for good jobs. Not every program completer wants a job; many will 
want to continue his or her education. The quality of the pro- 
grams taken in high school influences the ease of transition into 
postsecondary education. Guidance and counseling of students for 
vocational education continues to be one of the most important 
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support services in the schools. Placement services are another 
important area for support service improvement in vocational 
education. 

The portions of the model that focus on student recruitment 
and program completers deserve special attention. The students to 
be recruited should be those who can benefit most from vocational 
instruction, for example, academically talented students who need 
to explore career fields prior to making choices for postsecondary 
education. Mentally handicapped students who may be suffering a 
learning disorder are also included in this group. These students 
may need tangible feedback experienced through learning by doing. 
Recruitment of students requires coordinated effort among voca- 
tional teachers, administrators, and guidance counselors. The 
program completers portion documents the quality of the instruc- 
tional experience. Are students taking a series of courses in a 
particular program area? Why are students not completing courses? 
Are they leaving to take jobs? Are they leaving because the 
classes are not satisfying to them or their parents? The ques- 
tions need to be addressed if program planning is to be effective 
in meeting the needs of employers, students, and the community. 

Although teacher education and reeducation are not central in 
this model, teachers can significantly influence the success of 
vocational programs. The teacher, as manager of the instructional 
process, has the final word on activities provided for instruc- 
tion. A teacher who is current on workplace requirements can help 
students prepare for needed skills and available employment oppor- 
tunities. 

Similarly, parents have a great deal to do with the quality 
of vocational education provided by the schools. If funds are 
available, the chances of conducting a high-quality program are 
better than in poverty-stricken districts. An active involvement 
of advisory council members and community groups can support 
productive educational activities. 

Quality Based Education legislation in Georgia and the Carl 
D. Perkins Vocational Education Act at the national level have 
stimulated rethinking about how to deliver programs. Fifty-seven 
percent of the basic state grant from the Perkins Act goes to 
special needs programs in the states. Quality Based Education is 
resulting in a core curriculum that is likely to have the effect 
of increasing graduation requirements at the secondary level. 

It is no wonder, then, that educational product outcomes 
received major attention 'n the model, student knowledge and 
skills must be assessed in order to know the effectiveness of 
vocational programs. This information must be related to the 
characteristics of students coming into the program as well as the 
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ones exiting the program, either as program leavers or as program 
completers. The success of graduates in the market place is 
influenced by a number of other factors already mentioned by 
discussions of the context category. Such success indicators must 
be applied judiciously in the evaluation of vocational programs. 
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4. APPLICATIONS OF THE CONCEPTUAL MODEL 
TO VOCATIONAL EDUCATION 



Throughout this report, elements of planning and evaluation 
of vocational education programs have been discussed in various 
contexts. This section of the report will focus on common pro- 
cesses necessary to the operation of a vocational program. These 
processes deal with serving populations, scheduling classes, 
initiating courses, testing students, and validating programs. 
Next, a few thoughts are offered on implementation of the model. 
The report closes with "next step" questions. 



Local Program Questions 



The following selected questions are directly related to 
local program operations. Nevertheless, it is the need for 
excellence at this level that should influence state program 
improvement initiatives. The questions probably tend to be more 
relevant to planning decisions than evaluation, but data from one 
activity are used by the other. 



Whom should vocational Education Serve ? 

The advent of Quality Based Education and the Carl D. 
Perkins Vocational Education Act has put in focus the need for 
quality education with particular reference—in the case of the 
Perkins act — to special needs groups. The conceptual model 
described in this document suggests variables for planning and 
evaluation considerations, lor example, if a school district has 
targeted dropout prevention as a goal for the district, this 
model's characteristics of individuals may be useful in identify- 
ing students who are likely to drop out of school. If a district 
is interested in upgrading literacy training among adults, these 
same characteristics may be used to identify prospective adults 
for day and evening school classes. Significantly, one of the 
factors in the Virginia Polytechnic Institute study of vocational 
enrollment patterns was adult enrollment in secondary daytime 
vocational education programs (Frantz, Strickland, and Elson 
1986) . Displaced homemakers were present in many of the programs 
that reported increased vocational enrollment. 
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How Should Classes be Schedul et^? 



Increased graduation requirements have placed pressure on 
students who are trying to schedule two-period vocational classes, 
one of the very difficult problems that must be faced by a school 
administration is the need to schedule adequate time for students 
to get their work done in laboratory classes. This may lead to 
the more fundamental question of what job skills courses can be 
taught at the high school level and what courses must be taught as 
postsecondary instruction? A school district must make some basic 
decisions on what training it will offer. The role of an advisory 
committee from the community can be invaluable in assessing the 
need for skill development. The availability of education and 
training beyond high school may help to decide this question. 



What Cou rses Should be Offered? 

Community expectations for skill development and the 
availability of jobs also will influence the variables for the 
outcomes index used to evaluate the effectiveness of vocational 
programs at a given location. Low scores on the index could 
suggest the need for new programs or improved instruction. 

Secondary schools are decreasing their emphasis on skill 
development and have begun to offer more technology-based courses. 
These courses are adaptable to single-period classes. However, 
care must be taken that vocational education does not become so 
general and basic that the students fail to connect the competen- 
cies taught in class to work skills. 



What Achievement Tests should be Used? 

The availability of student occupational competency 
achievement tests raises the possibility of using these tests to 
measure student progress toward occupational objectives. The 
tests can be administered in puch as way as to protect the 
confidentiality of the student. Regional Education Service 
Agencies may or may not play a role in the administration and data 
processing of these tests. Generally, more control can be 
exercised over data processing activities and the accuracy of 
results if this operation were to be centralized. 

Traditionally, educators have been reluctant to use achieve- 
ment tests to compare programs in schools or to compare similar 
programs from school to school. The following are some of the 
reasons for this reluctance: 
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o The characteristics of some programs make comparisons 

difficult. For example, the differences between cosmetol- 
ogy and auto body work are so great that there are few 
measures common to both areas. 

o The lack of standardization in schools and programs 
mitigates against valid comparisons of outputs, one 
program may have an excellent reputation, so the students 
entering that program bring with them an aptitude for 
learning; another program in another location may not be 
so fortunate. Any comparison of programs must be 
accompanied by sufficient analysis to control for major 
sources of variation. 

o The results of achievement tests, once completed and 

identified with a school, have a tendency to become public 
knowledge. Even if the privacy of individual student 
scores could be protected, average achievement levels of 
classes or programs could be politically damaging to the 
teacher and the school staff if the results were made 
public. The notion of a public "report card" has not been 
acceptable in the past to governors, superintendents, and 
principals. 

Nevertheless, the technology exists for schools or states to 
collect achievement data on occupational programs. The National 
Occupational Testing Institute (NOCTI) in Big Rapids, Michigan, 
maintains a battery of student Occupational Competency Achievement 
Tests (SOCAT) for approximately 35 programs of vocational educa- 
tion at the secondary and postsecondary levels. Some states are 
using them to assess student skill proficiency and to award cer- 
tificates to students. The tests are adjusted for readability and 
contain a performance section as well as a written section. 
Students must be able to demonstrate practical knowledge of 
skills as well as pass a written test of competency. 



now Should Industry be Invol ved in the Validation of Programs? 

Industry validation for selected programs exists in the Trade 
and Industries (T&I) program field. The National Automobile 
Technicians Foundation (NATF) certifies high school programs in 
auto mechanics in eight speciality areas. A school must pass the 
review in three of the eight areas for a program to be certified. 
The capacity of the program to offer training is measured by (1) 
looking at the instructor's qualifications, (2) inspecting the 
equipment, and (3) examining the curriculum. Teams of three-five 
industry persons conduct the inspection over a 3-day period. 
Included in this review are conversations with employers of pro- 
gram graduates. *^ 
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There were 10 schools enrolled in this pilot program in 
Georgia in 1986; 4 earned certification. The program is comolete- 
xy voluntary. A similar validation activity is being developed in 
construction trades and the area of graphic arts in 1987. These 
types of validations offer the promise of better industry-related 
reviews in the future than have existed in the past for those 
vocational programs producing entry-level occupational graduates. 

Implemen tation of the Mod^ l 

There are many decisions to be made concerning the implemen- 
tation of this conceptual model. Not the least of these decisions 
is the trade-off between program reviews and site visits. Econo- 
mies of scale dictate a review of all programs when a program 
improvement team is on site. However, the visit may be triggered 
by low scores on outcome index indicators for fing program. Exact- 
ly how this problem will be resolved is uncertain at this time. 
There are several options, for example- 

o program reviews may be conducted as the need arises 

o program reviews may be held in abeyance until site visits 
to school districts can be scheduled 

o the outcomes index results may be treated as information 
only to be used in concert with other program improvement 
activities. 

The choice of options will be made by the Georgia Department 
of Education and will be based on results from the pilot test of 
the model, the degree of disruption a local district is willing to 
tolerate, and other factors. 

There are several other "choice points" in implementing this 
model. A few are listed below: 

o At what level should the data be collected? The sample 
unit should be determined by the need for information. 
For example, if gain scores for student achievement are 
needed, data on individual students are required, on the 
other hand, many program management questions can be 
addressed using averages from a class. 

o Should all data come from a management information system? 
Typically, MIS data are collected routinely, using system- 
atic processes, at minimum expense, should information 
for program planning be limited to readily quantifiable 
information? if the answer is no, then arr&ngements for 
using data from other sources need to be made. 
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o How should the data be transmitted from local districts to 
regional agencies and the state? The question of who 
should input the data into computers, at what location and 
how frequently must be addressed. Project GENESIS con- 
tains plans for terminals to be located in every school 
district in the state; however, a data collection system 
is needed prior to completion of Project GENESIS. An 
interim strategy for transporting data must be formu- 
lated. 

Ultimately, data from all school districts must be 
comparable, collected and processed at the least expense to the 
taxpayer. 

In general, when implementing innovations, it is better to do 
the following: 

o Think broadly and creatively before delimiting the imple- 
mentation plan 

o Proceed from the known to the unknown 

o start small and expand as the interest and capability of 
participants grow 

o Build on the strengths of the existing system 



Next Steps 

This document is the first of a series to be developed in 
support of a comprehensive planning and evaluation system for 
vocational education in Georgia, others will deal with the devel- 
opment of an outcomes index to be used in a desktop audit of 
programs, revision of on-site review questions and procedures, 
collection of data for the outcomes index, and an implementation 
plan and workshop. There are several questions that remain to be 
addressed in these documents. All of these questions will not 
necessarily be resolved during this project, but they bear on the 
quality of vocational education in Georgia. A partial list of 
these questions follows: 

o What public information is needed about the quality of 
vocational programs? 

o Do we need an Educational Development Plan (EDP) for each 
vocational student? 

o What contribution is secondary vocational education making 
to adult retraining for jobs? 



ERIC 



27 

.95 



o How are basic skills being provided to vocational stu- 
dents? 

o What are the common data needs between secondary and 
postsecondary vocational education? 

o To what degree should the vocational education planning 
and evaluation data interface with project GENESIS? 

o How can consximer-orientated evaluation reports be writ- 
ten? 



o What are the necessary interface relationships between the 
public schools and JTPA service delivery area? 

o How can comparison data with nonvocational students be 
obtained? 



o When should data be collected ftom the schools? 

These questions and others will be addressed in future documents 
produced by this project. 
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